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Abstract. This work focuses on improving state-of-the-art in stochastic local 
search (SLS) for solving Boolean satisfiability (SAT) instances arising from real- 
world industrial SAT application domains. The recently introduced SLS method 
CRSAT has been shown to noticeably improve on previously suggested SLS tech- 
niques in solving such real-world instances by combining justification-based lo- 
cal search with limited Boolean constraint propagation on the non-clausal for- 
mula representation form of Boolean circuits. In this work, we study possibilities 
of further improving the performance of CRSAT by exploiting circuit-level struc- 
tural knowledge for developing new search heuristics for CRSAT. To this end, 
we introduce and experimentally evaluate a variety of search heuristics, many of 
which are motivated by circuit-level heuristics originally developed in completely 
different contexts, e.g., for electronic design automation applications. To the best 
of our knowledge, most of the heuristics are novel in the context of SLS for SAT 
and, more generally, SLS for constraint satisfaction problems. 



1 Introduction 

Stochastic local search (SLS) 1111 is an important paradigm which facilities finding so- 
lutions to various kinds of hard computational problems via searching over a declarative 
formulation of the problem at hand. It has been recognized that one possibility to push 
further the efficiency of SLS techniques is to develop search techniques that exploit the 
structure of constraint satisfaction problems. Indeed, various structure-exploiting SLS 
methods have been developed (among others) for generic constraint satisfaction prob- 
lems (CSPs; for examples see 111211811011 ) and Boolean satisfiability (SAT; for examples 
see 1 1 612012 1 1 1 71 1 91 1 41 1 312214151 1 . 

This work focuses on developing efficient structure-exploiting SLS techniques for 
SAT. In more detail, we study techniques that are aimed at industrially relevant (or, 
as termed in the latest 201 1 SAT Competition, application) instance classes. The most 
effective methods for solving random SAT instances are based on SLS. Furthermore, re- 
cent advances in SLS for crafted SAT instances has resulted in an SLS method winning 
the satisfiable crafted instance category of the 201 1 SAT Competitior0. In contrast, on 
industrial instances the current SLS methods are often notably inferior to the dominant 
conflict-driven clause learning (CDCL) SAT solvers. 
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To the best of our knowledge, currently the best performing SLS method aimed at 
industrial SAT instances is CRSat 05161 . Instead of working on the rather low conjuc- 
tive normal form (CNF) level, CRSat searches for a solution directly on the level of 
arbitrary propositional formulas, relying on the compact representation form of Boolean 
circuits for a succinct way of representing propositional formulas. Furthermore, instead 
of relying on restricting search to input variables, as often has been proposed [ 16 20 121ll7fl9l , 
CRSat is based on the justification-based circuit-level SLS approach I14I13L search- 
ing over the whole subformula structure, and incorporates a limited form of directed 
circuit-level Boolean constraint propagation to further exploit structural aspects of the 
input formulas . 

We have recently shown that CRSat can be further improved by incorporating a 
structure-based heuristic for focusing search steps. This resulted in the depth-based 
variant of CRSat [6|. The depth-based heuristic has interesting fundamental prop- 
erties, including the fact that CRSat remains probabilistically approximately com- 
plete (PAC) [ 12] even when focusing search via the heuristic. 

Contributions The success of the depth-based search heuristic suggests that circuit- 
level structural properties of SAT instances can indeed be exploited to further im- 
prove SLS. Motivated by this, in this work we develop and experimentally study a 
wide range of novel structure-based SLS search heuristics, focusing on CRSat. We 
provide a systematic large-scale study of the proposed structure-based heuristics. We 
relate the heuristics to the depth-based heuristic studied in detail in [6|, with the aim 
of developing further understanding on what are the underlying properties that make 
the depth-based search work in practice. Furthermore, we investigate whether related 
(or even completely different) structural properties result in even more effective heuris- 
tics. Analysis of the experiments reveals various interesting observations on the type of 
structural properties of circuits result in effective search heuristics. 

Finally, as a future motivation for the studied heuristics, we are interested in ex- 
tending the CRSat approach, combining justification-based search over logical com- 
binations of constraints and exploiting limited constraint propagation, to more generic 
classes of constraint satisfaction problems (CSPs) for which local search is a very viable 
alternative B1I2I18I10L The development of good structure-based search heuristics for 
the circuit-level is directly applicable for the logical combinations of more high-level 
constraints, where the logical combinations can be viewed as circuits. 

Organization Key definitions and concepts related to Boolean circuit satisfiability are 
reviewed as necessary preliminaries in Sect. [2] Sect. [3] is dedicated to presenting the 
CRSat circuit-level SLS algorithm for which this work develops structure-based search 
heuristics. The heuristics are introduced in Sect.@] Before conclusions (Sect. [6]), results 
of an extensive empirical evaluation on the effectiveness of the structure-based heuris- 
tics are presented in Sect. [5] 

2 Preliminaries 

A Boolean circuit over a finite set G of gates is a set C of equations of the form 
g = f(gi, . . .,g n ), where g,g 1: ...,g n € G and / : {0, 1}" -> {0, 1} is a Boolean 



function, with the additional requirements that (i) each g G G appears at most once 
as the left hand side in the equations in G, and (ii) the underlying directed graph 
(G,E(C)), where E{C) = {(g',g) G G x G | g = /(... ,<?', ...) G C}, is acyclic. 
If (ff'i <?) £ E(C), tnen ff' i s a cAi'W of 5 and g is a parent of 5'. For a gate g, the 
sets of its children (i.e., the fanin of g) and parents (i.e., the fanout of 5) are denoted 
by fanin(C, g) and fanout(C, g), respectively. The descendant and ancestor relations 
fanin* and fanout* are the transitive closures of the child and parent relations, respec- 
tively. If g — f(jji, . . . , g n ) is in G, then g is an /-gate (or of type /). A gate with no 
children (resp. no parents) is an input gate (resp. an output gate). The sets of input gates 
and output gates in C are denoted by inputs(C) and outputs(C), respectively. A gate 
that is neither an input nor an output is an internal gate. Typical gate types include NOT 
(not(w) is 1 iff v is 0) and AND (and(zji, v 2 ) is 1 iff both v\ and v 2 are 1). 

An (truth) assignment for C is a (possibly partial) function r : G — > {0, 1}. A 
complete assignment r for C is consistent if r(g) = f(r(gi), . . . , r(g n )) for each 
9 = /(ffii ■ • ■ i5n) m C- When convenient we write (g,v) G r instead of r(g) — v. 
The domain of r, i.e., the set of gates assigned in r, is denoted by dom(r). We say that 
two assignments, r and r 1 , disagree on a gate g G dom(r) n dom(r') if t(^) 7^ 
For a truth assignment r and set of gates G C dom(r), let flip(G, r) denote the truth 
assignment obtained by changing the values of the gates in G, and leaving the rest of r 
unchanged. 

A constrained Boolean circuit C a consists of a Boolean circuit C and an assignment 
a for C. Each (<?, v) G a is a constraint, and g is constrained to w if (g, w) G a. 
A complete assignment r for C satisfies C a if (i) t is consistent with C, and (ii) it 
respects the constraints: rDa.lf some assignment satisfies C a , then C Q is satisfiable. 
A circuit that is not satisfiable is unsatisfiable. Without loss of generality, we assume 
that constraints are imposed only on output gates. 

The restriction t\q' of an assignment r to a set G' C G of gates is defined as 
{(g, v) G r I g G G'}. Given a gate equation g = f(g%, . . . , g n ) and a value v G 
{0, 1}, a justification for the pair (g, u) is a partial assignment a : {g±, . . . , g n } — > 
{0, 1} to the children of g such that f(r(gi), . . . ,r(g n )) — v holds for all exten- 
sions t D a. That is, the values assigned by a to the children of g are enough to 
force g to take the consistent value v. For example, the justifications for (g, 0), where 
g = AND(g 1 ,g 2 ), are {(51,0)}, {(g 2 ,0)}, and {(gi,0), (g 2 ,0)}, out of which the first 
two are subset-minimal. A gate g is justified in an assignment r if it is assigned, i.e. r(g) 
is defined, and (i) it is an input gate, or (ii) g = f(gi , . . . , g n ) G C and t | { 91 ,. . ., 9 „} isa 
justification for {g, r(g)). We denote the set of unjustified gates in an assignment r by 
unjust(G",r). 

3 CRSat: Justification-Based SLS with Forward Propagation 

CRSat is an SLS-based SAT algorithm for Boolean circuits that operates directly on 
circuit structure - that is, without the conversion to CNF. The algorithm was first de- 
scribed in [5] and was subsequently analyzed theoretically and improved in (6). In this 
section we provide a high-level overview of the algorithm, and refer the reader to [ 5 6 1 
for additional details. 



CRSat is based on the justification-based II 141 131 approach to circuit-level SLS. 
In this approach, the circuit is traversed from the outputs to inputs, and the values of 
the internal gates are adjusted using local information in an attempt to eliminate all 
unjustified gates. CRSat combines a weakened version of justification-based SLS with 
so called limited forward propagation - a restricted form of circuit-level Boolean con- 
straint propagation, described in what follows. 

Pseudo-code for CRSat is presented as AlgorithmQ] First, a complete extension of 
a random value assignment to inputs(C Q ) is constructed, i.e., the value of each uncon- 
strained internal gate is set consistently with the values of its children. Then, as long as 
unjust(C Q , t) is not empty (i.e., r is not a satisfying assignment), the algorithm heuris- 
tically selects an unjustified gate g (line [61 we will discuss gate selection heuristics in 
the next section in detail). Once an unjustified gate g is chosen, the algorithm selects 
a justification u for (g, r(g)) (lines [71- [T3l> and performs a search step. The latter con- 
sists of (i) flipping the values of gates on which a and t disagree (line[T5l), followed by 
(ii) propagating the consequences of the flip towards the outputs of the circuit (hne[T6ll. 



Algorithm 1 Generic CRSat(C q , wp, cutoff) 
Input: C Q - constrained Boolean circuit 

wp - noise parameter ,i.e., probability of random walk 
cutoff- cutoff, i.e., maximum number of steps 
Output: status - SAT if a satisfying assignment for C a is found, UNKNOWN otherwise 

r - a satisfying assignment for C a if found, otherwise 
1 : r a complete extension of a random assignment to inputs(C Q1 ) 
2: steps <— 

3: while steps < cutoff do 

4: if unjust(C Q ,r) = then 

5: return (SAT, r) 

6: g <— a heuristically selected gate from unjust(C a , r) 
7: £ «— the set of justifications for (g, r(g)) 
8: with-probability wp do 

9: a <— random element of E > random walk 

10: otherwise 

11: a -s— a random justifications from those in £ that minimize 

12: the number of unjustified gates after the step > greedy downward move 

13: end with-probability 

14: G <— set of gates in a that disagree with r 

15: T^flip(G,r) oflip 
16: r <- LBCP-FORWARD(C a ,G,t) > limited forward propagation 

17: steps <— steps + 1 
18: return (UNKNOWN, 0) 



The justification a used to make a step can be selected from the set S of all justifica- 
tions for (g, r(g)) either at random (with probability wp), or greedily with the objective 
of minimizing the number of unjustified gates after the step. Note that taking S to be a 
set of subset-minimal justifications results in good performance in practice; this is also 
how our current implementation works. 



The forward propagation procedure LBCP-FORWARD is presented as Algorithmic] 
It uses a priority queue Q of gates (with no duplicates) that allows to query the small- 
est gate according to a topological order in constant tim^l Essentially, the procedure 
implements a circuit-level Boolean constraint propagation algorithm, except that (i) the 
values are propagated only towards the outputs of the circuit, and (ii) propagation along 
each path stops immediately when an unjustified gate becomes justified; hence it im- 
plements limited forward propagation. The addition of limited forward propagation to 
justification-based SLS results in multiple orders of magnitude speed-ups on industrial 
SAT instances |3l . 



Algorithm 2 LBCP-Forward(C q , G, t) 
Input: C a - constrained Boolean circuit; 
G - a set of gates whose value changes are to be propagated. 
t - an assignment for C a \ 

Output: t' - an assignment for C a which is a result of limited forward propagation of the 

assignment t\q. 
1: t' <- r 
2: Q.ENQUEUE(G) 

3: while -> Q. empty do 

4: g <-Q.POP_FRONT 

5: if g € G then > g is one of the original gates 

6: Q.ENQUEUE(fanout(g)) 

7: else 

8: if g € unjust(C a , r') \ dom(a) then > g unconstrained and unjustified 

9: r' ^flip({g},r') 

10: Q.ENQUEUE(fanout(g)) 

11: return r' 



It comes as no surprise that the effectiveness of CRSat depends critically on the 
way the gates are selected for justification during the search (Line [6] of Algorithm [TJ. 
A good selection heuristic focuses search to the most important gates in terms of sat- 
isfiability. On the other hand, if a too deterministic (focused) selection procedure is 
used, the search may not converge into a satisfying assignment. In [6| we showed that 
the efficiency of CRSat can be significantly improved by focusing the search using 
a structure-based gate selection heuristic which takes into account the depth of the se- 
lected gates. In the next section we describe a number of additional structural properties 
of gates and propose a number of gate selection heuristics based on these properties. 

4 Structure-Based Search Heuristics for CRSAT 

In this section we introduce a number of heuristics for selecting of the unjustified gate 
to justify at each search step in the main loop of CRSat (line|6]of Algorithm Q}. The 
underlying idea is that these heuristics should be able to take into account the structural 



4 Recall that a topological order on the set of gates in a circuit is any strict total order < that 
respects the condition "if gi £ fanin^), then gi < 32" 



properties of the constrained Boolean circuit at hand, and focus the search on the gates 
that are deemed important based on these properties. Additionally, we must aim at effi- 
ciently computable heuristics, as the main loop may be executed millions of times in a 
typical run of the algorithm (although, in contrast to typical SLS algorithms, most of the 
computation effort in CRSat is attributed to the execution of forward propagation, and 
hence we can afford slightly more expensive computations than usual SLS heuristics). 

We now give a listing of the initial set of gate properties, with intuition on why 
these properties may be interesting. We then describe the corresponding gate selection 
heuristics, and, in the next section, present the results of the preliminary empirical eval- 
uation of these heuristics. The analysis of the results will lead us to the development of 
additional heuristics, which will be described and analyzed in Sect. [5] 

Depth: depth (C, g), where the depth of a gate g in C is 



The importance of gate depth for CRS AT was justified theoretically and confirmed 
empirically in |6|. The key aspect is that selection of gates with high depth drives 
the algorithm close to the inputs of the circuit, thus allowing the algorithm to ex- 
plore the space of assignments to input gates faster^ The depth of all gates in C 
can be computed in 0(|C|) time (where |C| denotes the number of gates in C), and 
stored for constant time retrieval. 
FO: |fanout(C, 5 )| 

Gates with large fanout size are in a sense more influential than the rest. Intuitively, 
by forcing CRSat to justify these gates, the truth values of these critical parts of 
the circuit are fixed first, which may result in many of the other gates' values to be 
set by forward propagation. The fanout size of a gate is retrieved in constant time. 



This is also a measure of the influence of the gate in the circuit: intuitively, the 
larger the size of the transitive fanout, the more influence the gate's value has on 
transitively justifying the output constraints of the circuit via forward propagation. 
The computation of the size of the transitive fanout of a gate requires 0(\C\) in the 
worst-case (although typically only a fraction of gates in C have to be evaluated). 



The size of the transitive-fanin of a gate g can be considered an estimate of the 
number of search steps required to justify all gates in the sub-circuit rooted at g. 
This measure is also related to the size of the interest set used as an objective 
function in justification-based SLS algorithm BC SLS 11 141 1 31 . The computation 
of the size of the transitive fanin of a gate requires 0(|C|) in the worst-case. 
CC: CC(C, g, r{g)), where the SCOAP (Sandia Controllability and Observability Anal- 
ysis Program) combinational controllability measure [9 | CC is defined as follows: 

5 Here one should notice that driving the search towards input gates in justification-based search 
is different from the idea of restricting the flips to input gates as in |16|20J.21|17| 19| due to the 
conceptual differences of these approaches. 




if g 6 outputs(C a ) 

1 + max{depth(C, g') \ g' € fanout(C, <?)} otherwise. 



TFO: (fanout* (C,g)\ 



TFI: |fanin*(C,s)| 



CC(C, <?,()) 



1 if g E inputs(C) 

1 + muv efanin(c ., 9 ) CC(C, g' , 0) if g is an AND-gate, 



CC(r a D- l l ifgGinputs(C) 

' j ~ I 1 + £ S ' e fanin(C, S ) CC(<7, <?', 1) if fl is an AND-gate. 

Given a gate 5 and its current value v g , SCOAP aims to provide a measure of how 
difficult it is to satisfy the sub-circuit rooted at g given that g is constrained to v g 
(i.e., to control the value v g at g). Originally, SCOAP was used as a combinational 
testability measure. For our purposes, SCOAP intuitively provides a measure of 
how difficult it is to transitively justify the output constraints of a circuit. Due to 
the fact that we apply And-Inverter graphs (AIGs) as benchmark instances in this 
paper, the definition is restricted to AND-gates only. However, the definition can be 
naturally extended to other gate types. 

Here one should notice the original definition of SCOAP assigns for NOT-gates 
(negations) the value of the gate's child incremented by one. In contrast, here we do 
no increment such values, but instead implicitly skip NOTs in the following sense. 
In case g = NOT(</), all gates in fanout of g' are included in fanout of g' instead 
of g. This is due to the fact that negations (inverters) are handled implicitly in the 
justification steps and forward propagation performed by CRSat, and hence the 
CC value assigned to each NOT-gate equals the value assigned to the gate's child. 
Note that SCOAP controllability measures for all gates in C can be computed in 
0{\C\) time. 

CO: CO(C, g), where the SCOAP combinational observability measure [9] is defined 
as follows: 

CO(Cq) = (° ifg G outputs(C) 

1 1 + min 9 / efanout (c, s ) CCO(C, g\ g) otherwise, 

where for an AND-gate we have 

CCO(C,</,<7) = CO(C</)+ Y, CC(Cfl",l). 

gr»efanin(C, S ')\{s} 

As in CC, we implicitly skip NOTs in the definition. This measure attempts to cap- 
ture how difficult it is to observe a specific value for a gate given the output con- 
straints; in other words, how likely is it that the value is part of a minimal justifica- 
tion that is transitively consistent with the output constraints. The measure can be 
computed for all gates in Cm 0(\C\) time. 
Flow: flow(C, g), where the output flow value of a gate g in C is 

!1 if g 6 outputs(C) 

Y , flow ^g') , otherwise. 



In other words, we compute a total flow value for each gate by pouring a unit 
quantity flow down from the output gates of the circuit. Here it is important to 



notice that the definition of flow implicitly skips NOT-gates. This flow-based idea 
was first evaluated in 1 1 5 1 as a heuristic for restricting the set of decision variables in 
CDCL solvers. Our intuition is that, if a large total flow passes through a particular 
gate, the gate is globally very connected with the constraints in r, approximating 
in a sense the number of possible paths for forward propagation, and thus g would 
have an important role in the satisfiability of the circuit. 

Each of the structural properties presented above gives rise to a pair of gate selection 
heuristics: for a given property f(C a , g, r), one heuristic selects at random a gate from 

argmax f(C a ,g,r). 

g£unjust(C Q ,r) 

We will refer to this heuristic as a max-variant, /-max, of the heuristic based on /. 
And, a dual heuristic, the min-variant, /-min, selects at random a gate from 

argmin f(C a ,g,T). 

pEunjustfC" ,r) 

Thus, we have 7 pairs of dual heuristics, and the baseline heuristic Rand that simply 
selects a random gate from from u nj ust(C Q , r ) - this is the heuristic used in the original 
paper on CRSat 0. 

We now note that some of the presented structural measures of gates are in parts 
correlated (either positively or negatively) with gate depth (these are TFO, TFI, CC, 
CO), while others (FO, Flow) are not. The reason that we pay a particular attention to 
the depth is that we know that the Depth-max heuristic is very effective [6|. As such, 
when we evaluate the heuristics based on the properties that are positively correlated 
with depth (depth-friendly heuristics) we are interested in detecting improvements over 
Depth-max. Such an improvement would suggest that another, perhaps more funda- 
mental property, is at play in CRS AT-style circuit SLS. Furthermore, the duals of depth- 
friendly heuristics are expected a priori to perform poorly. In evaluating the heuristics 
that are not correlated with depth (depth-agnostic heuristics), we are also interested in 
detecting significant differences in performance on some classes, or even on particular 
problem instances. Such differences would suggest that depth-agnostic heuristics might 
be used as secondary heuristics in CRSat (e.g. for tie-breaking). 

To summarize, the following heuristics are the primary targets of the empirical eval- 
uation and analysis presented in the next section: 

- Baseline: Rand and also Depth-max. 

- Depth-friendly: TFO-max, TFI-min, CC-min (small controllability value means 
the gate is easy to control, and hence intuitively close to inputs), CO-max (large 
observability value means the gate is difficult to observe, and hence intuitively far 
from outputs). 

- Depth-agnostic: FO-min, FO-max, Flow-min, Flow-max. 

5 Evaluation 

In order to provide an objective empirical comparison of SLS solvers, the well-known 
SLS textbook by Hoos and Stutzle [ 1 1 1 suggests a procedure for finding near-optimal 



noise (the setting of the parameter wp in Alg.Q]) by essentially binary searching for the 
noise values for each individual instance and solver to be evaluated. While full binary 
search is computationally infeasible given the vast number of benchmark instances used 
in our experiments and, on the other hand, the computational resources available to us, 
we applied the following approximation of the Hoos-Stutzle scheme. Noise was opti- 
mized for each solver and instance individually based on 25 tries using a timeout of 200 
seconds per try (with no limit on the number of steps), at noise values 0.05, 0.1, 0.2, 
0.3, 0.4, 0.5. The noise with highest success rate (primary criterion) and best median 
time (secondary criterion) was selected. In cases where there were two or more options 
ranked best using both of these criteria, a random candidate among those options was 
picked. Note that the benchmark-class based noise optimization, which is computation- 
ally cheaper, is often insufficient on industrial application benchmarks. For example, 
among 61 solved instances of one of the benchmark classes described below (sss-sat- 
1.0) we found 10 instances to have a near-optimal noise value, wp no , of 0.05, 10 with 
wpno = 0.1, 14 instances with wp no — 0.2, 9 instances with wp no — 0.3, 9 instances 
with wp no = 0.4 and 9 instances with wp no = 0.5. 

The reported CPU times and number of steps for each instance are the median CPU 
time (in seconds) and the median number of search steps with the best noise setting 
over 25 tries for the experiments summarized in Fig. 1 and 3, and over 100 tries for 
the experiments summarized in Fig. 2.@. The experiments were performed on an HPC 
cluster, each node of which runs on a dual quad-core Xeon E5450 3-GHz with 32 GB 
of memory. 



5.1 Benchmark Families 

As benchmarks, we considered over 650 And-Inverted circuits (AIGs, that is, con- 
strained Boolean circuits in which gate types AND and NOT are used) from five dif- 
ferent industrial application benchmark classes. We obtained the AIGs as described in 
the following. 

hwmcc08-sat 204 satisfiable AIGs obtained from the Hardware Model Checking Com- 
petition 2008 problem^. The original sequential circuits were unfolded using the 
a i gt obmc tool (part of the AIGer package-^) The step bound k = 45 was used for 
the time frame expansion. 

smtqfbv-sat 61 satisfiable AIGs generated by using the Boolector SMT solved O to 
bit-blast QF_BV (theory of bit-vectors) instances of the SMT Competition 2OOS0 
into AIGs. 



Based on our experience, given the large number of instances, 25 tries is enough to detect the 
main trends. The experiments described in Fig. 2 require more precision. 

7 Original instances available at |http : / /fmv . jku ■ at /hwmcc08 /| 

8 Available at |http: //fmv. jku ■ at/aiger/| 
9 |http : / /fmv . jku . at /boolector /| 

' |http : / / www . smtcomp .org/2009/| 



sss-sat-1.0 98 satisfiable AIGs from "formal verification of buggy variants of a dual- 
issue superscalar microprocessor'^ [23 1 . These circuits, originally in the ISCAS 
format, were converted to AIG using the ABC systerrQ |7 1. 

vliw-sat-1.1 98 satisfiable AIGs from "formal verification of buggy variants of a VLIW 
microprocessor", originating from the same place and converted to AIG in a similar 
fashion as sss-sat-1.0 instances. 

sat-race Satisfiable AIGs filtered from a total of 200 instances used in the final round 
of structural SAT track of the SAT Race 2008 and 2010 competitions 

In order to be able carry out the experiments in practice, we picked a selection of a 
total of 300 instances from these benchmark classes as follows. Based on the good per- 
formance reported in [6| for the Depth-max heuristic, we filtered out trivial instances 
for Depth-max (instances for which the median number of steps was < 730). From 
the remaining ones, in order to we picked those instances that we consider solved by 
Depth-max (i.e., instances for which the success rate for Depth-max was > 50 %l 14 [ 
This resulted in the following distribution of instances: hwmcc08 - 95, smtqfbv - 46, 
sss-sat-1.0 - 61, vliw-sat-1.1 - 96, and sat-race - 2. 



5.2 Results and Analysis 

Fig. [T]presents a "cactus" plot, i.e., the number of instances that can be solved within a 
given timj^l summarizing the comparative performance of the 15 structure-based gate 
selection heuristics described in Sect. H] The following conclusions can be drawn. 

First, we note that whenever a heuristic outperforms the baseline Rand heuristic, its 
dual performs worse than Rand, and vice versa. In fact, we see that in many cases the 
better the performance of a heuristic, the worse is the performance of its dual. This sug- 
gests that the properties proposed in Sect. [4] are meaningful in the context of CRSat. 
One exception to the nice "symmetric" picture is the pair based on SCOAP combina- 
tional controllability CC, where the worse of the duals, CC-max, performs surprisingly 
close to the baseline Rand - we will discuss this point later. An additional observation 
is that the depth-friendly heuristics TFO-max, TFI-min, CC-min and CO-max al- 
ways perform significantly better than their duals, and, furthermore, form most of the 
best performing heuristics. This corroborates the hypothesis that there is an important 
underlying property correlated with the depth of gates. 

Second, we observe surprisingly good performance from the depth-agnostic Flow- 
min. Recall that, intuitively, gates with high flow are those that have large influence on 
other gates in the circuit. Thus, on the surface, this result casts doubt on the role of the 

11 Available at |http : / /www .miroslav-velev ■ com/ sat_benchmarks ■ html| 
12 |http : / /www . eecs . berkeley . edu/ ~alanmi/abc/| 

13 Available at |http : / /baldur . iti . uka . de/sat-race-2010/downloads ■ html] 

14 This allowed us to perform these extensive experiments in practice within the given time frame. 
We hope to extend the experiments also to those instances unsolved by Depth-max. 

15 The median CPU times were used for the plot. The median number of search steps would also 
be an appropriate measure for comparing the quality of search heuristics. However, the relative 
performance differences based on time and on number steps are very close in this case, and 
the cactus plot using running times is easier to read. 



Fig. 1. A comparison of the performance of 15 gate-selection heuristics described in Sect.|4]as a cactus plot, i.e., the number of those instances that can 
each solved within a given time limit. An instance is considered solved if a success rate over the 25 tries is > 50%. The CPU time of a solved instance is 
the median CPU time for the instance over all runs (including the unsuccessful ones). 
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Fig. 2. Scatter plots that compare the performances of selected heuristics in terms of the median 
number of steps, over 100 tries. Timed-out instances are plotted with the number of steps set to 
10 7 , on the vertical and horizontal lines. 



influential gates in the context of CRSat. On the other hand, between the two duals 
based on the size of the fanout of gates, it is the FO-max that performs well, rather than 
FO-min. A closer look at some of our instances resolves this apparent contradiction 
- the flow is not depth-agnostic, but, in fact, is negatively correlated with depth. The 
reason for this is that most of our benchmark circuits have significantly more inputs 
than outputs, and thus gates that are close to inputs tend to have small flow values. At 
the same time, we did not detect any interesting relationships between Depth-max and 
FO-min, most likely due to the fact that the latter is much more a local property than 
the former. This suggests that to further study the effects of "influence" of gates in the 
context of CRSat, alternative measures are needed, e.g., ones that are based on graph- 
theoretic centrality measures. This conclusion is corroborated by the fact that, although 
the depth-friendly heuristics capture high influence — gates with large depth often have 
large transitive fanout and thus have high influence through forward propagation — the 
results show that TFO-max is not the best performing heuristic. 

Finally, we observe that the SCOAP-based heuristics CC-min and CO-max, as well 
as the TFO-max heuristic based on the size of the transitive fanout of gates, do not per- 
form as well as Depth-max. However, in contrast, the heuristic that prefers gates with 
small transitive fanin, TFI-min, appears to perform noticeably better than Depth-max. 
The scatter plot in Fig. |2(a)| which compares the performance of these two heuristics in 
terms of the number of search steps demonstrates that the size of the transitive fanin of 
gates can provide a better guidance to CRSat than the depth of the gate. 

Note that gates with small transitive fanin are very likely to be close to the inputs. 
Based on the theoretical analysis of CRSat in (5) and [6| the performance of the algo- 
rithm should improve if it arrives to the input level frequently. Hence, to get insight into 
the reasons of the good performance of TFI-min, we need to understand whether the 
heuristic is effective simply because it brings the algorithm faster to the input level, or 
whether there is another mechanism at play. One way to investigate the answer to this 



question is to compare the performance of TFI-min with a heuristic that is based on a 
measure that disregards the number of gates in the sub-circuit rooted at the gate, and 
takes into account only the distance from the gate to the input level. Such a measure, 
well known in EDA literature, is called the level of a gate, and is defined as follows: 

Level: level (C, g), where the level of a gate g in C is 

levelfC g) = <f ° if. 9 G inputs(C*) 

v ,JI \ 1 + max{level(C, g 1 ) \ g' G fanin(C,g)} otherwise. 

Thus, level (C, g) is simply the maximum distance from the gate g to an input gate, 
and so the depth-friendly heuristic based on level, Level-min, would control the search 
solely based on the distance to the inputs. 

The comparative performance of TFI-min and Level-min is presented in the scat- 
ter plot in Fig. |2(b)| We observe that performances of the two heuristics are highly 
correlated. As such, this comparison does not give a definitive answer to the question 
of which measure is more fundamental for CRSat. To gain some insight, we can in- 
troduce heuristics that go for the input gates more aggressively than Level-min. Such 
heuristics can, for instance, be based on the following measures: 



LLevel: level (C, g), where the "low" level of a gate g in C is 
Meve](C,flO 



if g G inputs(C) 

1 + min{llevel(C, g') \ g' G fanin(C, g)} otherwise. 

ALevel: alevel(C, g), where the "average" level of a gate g in C is 

JO if 5 G inputs(C) 

alevel(C,.g) - ^ + 5^.^ | e vel(C <?')/|fanin(C, <?)| otherwise. 

Thus, the "low" level of g is the shortest distance from g to some input gate, while 
the "average" level of g is somewhere in between the level and the "low" level; that 
is, we always have level(C, g) > alevel(C, g) > llevel(C, g). As such, the LLevel- 
min heuristics will drive the search to the input gates extremely aggressively, while the 
ALevel-min heuristic represents a middle ground between Level-min and LLevel-min. 

The cactus plot in Fig.[3]summarizes the comparative performance in terms of CPU 
time of the three level-based heuristics described above and TFI-min. We note that the 
performance of level-based heuristics degrades as the heuristics attempt to drive the 
search towards the inputs more aggressively. This observation provides partial evidence 
to the hypothesis that the size of transitive fanin of a gate, which provides an estimate 
of the amount of work needed to justify a sub-circuit rooted at the gate, is a more 
fundamental structural property in the context of CRS AT. However, in order to evaluate 
this hypothesis properly, we need to discover classes of problems where the measures 
Level and TFI are not correlated. Finally, due to the fact that on the instances in our 
benchmark set the two measures appear to be correlated, we note that since Level is a 
cheaper-to-compute measure, in practical applications one might want to consider using 
Level-min, rather than TFI-min, as a gate-selection heuristic. 
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Fig. 3. A comparison of the performance of TFI-min with various level-based gate-selection 
heuristics as a cactus plot, i.e. the number of those instances that can each solved within a given 
time limit. An instance is considered solved if a success rate over the 25 tries is > 50%. The 
CPU time of a solved instance is the median CPU time for the instance over all runs (including 
the unsuccessful ones). 

6 Conclusions 

We presented results of experiments on the applicability of different circuit-level prop- 
erties as the basis of structure-based search (gate selection) heuristics for the state-of- 
the-art SLS method CRSat for industrial-related Boolean satisfiability instances. The 
results can be seen as first steps towards understanding the role of structural information 
in justification-based local search for SAT with limited Boolean propagation integrated 
into the search. We identified a number of easy-to-compute structural properties which 
appear suitable as the basis of heuristics for CRSat, some of which can even outper- 
form the recently introduced depth-based variant of CRSat. The promise of the result- 
ing heuristics was also corroborated by showing the dual properties result in extremely 
weakly performing heuristics. 

The now presented results open up various interesting questions for further work on 
improving structure-based SLS for SAT. First, the observation that somewhat differently 
defined structural properties result in good heuristics suggests to study different ways of 
combining the resulting heuristics for achieving even better performance. This includes 
the question of what are the actual underlying properties to give good performance, and 
which the now studied easy-to-compute properties may be approximating. In addition 
to gate selection heuristics, we also aim to study different objective functions that are 
based on structural properties of SAT instances. Finally, we note that the development 
of good structure-based search heuristics for the circuit-level is directly applicable for 
the logical combinations of more high-level constraints (more generic CSPs), where 
the logical combinations can be viewed as circuits. This is one of the main research 
directions we are currently pursuing. 
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